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The persistent and hydra-headed attempts in ^jthe past to improve el e-: — 



nto 



mentary instruction may be roughly divided in^o two unequal parts'. The 
smaller number of innovations ^aimed .at broadly conceived teacher development, 
as attempting to changi teachers and their instructional ways by enriching 
their understanding -of subject mattei/, improving their diagnostic .and man- 
ugerial* skills, or increasing the repertoire of their interactiotis with 
students. The affective education movement, the Academic Year linstitutes, 
and the open education advisory services are examples of 'such efforts. By ^ 
far the greater number of movesr toward reform were those that sbught to 
bypass the teacher, seeking instead to affect students through jthe mediation 
of curricular fhaterials or activities. Thus the curriculum reform movement 



engaged the issue of teacher development only to the 



implementation of new materials was seen to depend on some conl|:omitant train- 
ing of the teacher. ' Of course, neither approach ever came to pass in, pure 
form, but a careful scrutiny of goal statements and' patterns of res.ource diis- 
.tribut'ion reveals the dev'elopers' leanings. The failure of materials intended 

\ raw ' ' ' ^ »■ . 

to be "teacher-proof" is 'a tale so o'ften repeated as to require no retelling - 

^ . . u 

here. There is no plethora of successful teacher development efforts to point 
tG', except to note that thi^ effort never received tne same attention and 
largesse of resources as did the area of curriculum development. 



exteAt that productive 



. The PLATO Elementary Program 



^ The undertaking we shall talk about here is one we have been . observing 
and evaluating for close- on to two years. It h^d the| development of curriaular 
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^ ^ - ... - • V . . 

materials, in this case programs delivgred over an interactive computer ^.^ * - 

^ ' ' ^ * ' • ,^ 

system, arf its main objective* TfTe deVelopers, ,hodever, professed a view 



of the teacher's role a^ critical to the effective use of these programs. 

' ** * « 

" ( ' ^ 

Under optimal conditioifs the materials would be integrated with ongoing 

cirrriculum, mo4ifiedS>y the classroom context, even shaped by, the needs, ^ 



goais'li^ and style of the teacher. thj.s project thus^sought to combine 



is\sougnc CO comoine ^ 
* / educative functions for students and teachers, albeit with far more artic- 

ul^ited plans for the instruction of students, with teacher educatibn largely 

« J* 

} 

. serving program im^^lementatfon. 

- - the Pl^TO project is an ef fort unique "in scope and^ character / a 

* brief description of 1/ts 'main features is offered to frame our subsequent^ ^ 

c ^ " ^ ' . .\ ^ 

^ observations. The PLATO elementary reading -and mathematics demonstrations 

/ ^ *» 

represent ambitious attempts at sequential and concurrent hardware., /software, 

' . *^ - ; / ^ • 

^ curriculum, and implementation development of tutorial computer assisted 

instruction in elementary schools. '.Since 1960, the engineering and systems 
development of the PLATO system and TUTOR teaching language has been under 
way at the Computer-Based Education RiBsearch Laboratory of the University 
of Vlllinois, with much in-house experience acquired over the years in authoring 
^ "liniversity- level lessons and sophisticated simulations and games. While some 

■■ ^ ■ ■ ' c ^ 

.eflementary mathematics .unitis had already b^en developed for earlief'^versions 
of PLATO; it was only in 1972, with the awarding of funds from the National 

.Science Foundation, that development of PLATO In lesson sequences /covering 

J 

significant portions of the beginning reading and 4th to 6th-grade elementary 
mathematics curriculum was undertaken. The PLATO system, with nearly 1,000 



terminals connected to a CDC Cyber 70 compubervin Urbana, iff dble to .make 

v< » • , ' 

a substantial library of lessons available to any user on- call, constrained 
only by the system's extended core storage capacity. Thj^ PLATO terminal is 
^ device with a typewriter- J.ike keyboard, a "plasma*^ screen, internal 
character memory, Itnpressive graphic and slow animation capability, rear, 
projection of color microfiche images, computer-conJferolled random access 
to disc-stored audio messages, and. the' ability to sense the portion of the 
display toucf^ed by the student. Although' develojiment and imp.jrovement 4 ' 
continues on hardware and system features, the system is now sufficiently 

^stable to permit the orderly introduction of the stlH-developlng curriculum 
Intp the elementary school classrooij. 

The many unique features of the project should i\pt mask the fact 
that the development and implementation issues emerging from the evaluation 
have relevance beyofid the specific medium of insti*uction in* use, the means 
cheapen to carry them out,^,and the actual course of implementation. A good 
understanding of the developers' conceptions and intentions can ptpve enlight 
ening about innovative cufricular programs across a range of conditions. 

We should like to describe the iPsues of teacher selection, orientation, 
and early support in some detail. ' These are common concerns in most implemen 
tations of curricular change. They can, . however , -be handled or solved in 

•'a variety of ways, i^epending on the implemen ter ' s convictions, goals, skills, 

aad situational constraints. The strategies deployed by the PLATO project 

^ ^ It 

will serve to illustrate some of the assump^ons and consequences of these 
tactics, which shall fie examined ^h ere. ' 



... -4- 



While we will be sharing some tentative observations abx>ut the 
first phase of the, project, the question persists of . how an effort of this 
scope and complexity may be appropriately assessed. The meaningful questions 

" to ask| processes to observe, roethoddlogies and tools to use, audiences 
to address, remain open issues. Our impressions and the assessments ye 
will make are b^sed on varied informal sources of information — they 

/were culled from documents, telephone conversations, formal meetings, and 
even encounters at 'the ;coffee urn. Our formal systematic data collection 
was done via five avenues: josx in-cfepth interview of teachers, classroom 
observations, system data, teacher logs, and norm-and content-referenced 
attitude anj achievement tests. \^ 

The teacher interview, semi-structured and open-ended in format, sought 
to gain access to the teachers ' pedagogic constructs, especially those 
related to math or reading. We also solicited the teachers' perceptions 
of their own classrooms, the teaching role, children as learners, and 
related matters that were fudged relevant to the utilization of a new teaching 
aid. The teachers were also probed |or their expectations of and j)redispQsition8 

toward the new resource. 

/ " . ' . 

The inteifView had been originally designed by the Early Education Group 

^ < 

at ETS for a study of teachers working in open-education settings. This 

k • 

instrument was revised with the needs o^ the present evaluation in mind, 

0 

with the addition of PLATO-related questions, as well as extensive probes 
ofsthe teacher's conceptions of math and reading* Only a small portion? of 
the interview data will be reported here; i.e., information primarily dealing 
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with the teachers' entry into the program and their perception of early . * 
orientation efforts. ^ • ^ c 

The rationale and the development of the means for classroom obser- 
vations will/be described in detail in the- next section of this paper, thus 
setting the context for the investigation of mode of PLATO use and nature 
of classroom integration. ' v 

The log, although not kept by all teachers, and irregularly by those 
who did keep it, yielded valuable information, from the teachers' per- 
spective, on life with a neonate innovation. The <Jata yielded by the logs 
will not be attributed to their specific source.^ 

\ 

\ * * 

Selection of Participants . 

One *of the early decisions program developers need to make is to identify 

and select those who will try out the innovation. The users of curricular 

* 

materials are school districts, i.e., superiatendents^ principals, teachers, 
pupils, yand parents. While the teachers and pupils are most directly affected 
by new programs, administrators and parents are recipients of important 
secbadary benefits or losses, and at times have significant input into dec- • 
isioifs of acceptance or rejection. 

After initial negotiations with more distant urban and rural school 
systems, the directors of the elementary PLATO projects took a consequen- 
tial step when, they approached the two school districts adjacent td the 
University where the program was being developed. Several advantages ^ccrued 
to this choice. The districts had a long histroy of University connections; 



1. We will not report on tes|t data in this paper. 

I 



/ 




-6- 



they had often served as testing ground for a number of previous, and * «S99R 

ongoing projects. The schools were accustomed to, or at least fHamillaf ^ 
with, the disruptions and Intrusions that accompany pilot projects. Ready 
access to the schools made Information flow between the user and project 
staff with greater frequency , richness, and shorter turn-around time. 

Sch6ql personnel were not unaware of the benefits, direct and derived*, 
that cooperation with the University and some of Its renowned educators 
. could bestow. In addition to the direct benefits of "free" access to an 
expensive and locally hlgh-.status tesource, there were the less tangible 
rewards of stature by association ^d,oOn occasion, the availability of 
educational and material resources that were only tangenfially related to 
project needs. On ba^^ance, however^ it appears that the association at 
this stage ben&flts PLATO staff more than the schools* A tacit recognition 
of the debt' Incurred Is'sometlmes made manifest ln*^rdject decisions about 
dls trlbutl'On of resources. As an example, even in classes where the terminals 
are judged to be used only marginally, PLATO staff have not made unilateral 
decisions to remove the^ resource if the teacher wants it and has invested 
energy in Incorporating it into the classroom. 

The convenienoe of trying out Ideas, methods, and materials in one's 
own backyard may, however, be offset by other consequences o£ this strategy. 
Thja easy and frequent access to the implementer may foster dependency on 
the part of the user, who"^then does not invest the requisite effort in 
acquiring facility with the resource, thus giving the evaluator a false 
Impression of the cost invdlved in assimilating it into the classroom. 
The Implementor on the other hand, is not pressed to articulate and develop, 

'8 . . 



in publicly accessible and Exportable forms, the skills and knowledge 
necessary for the effectl^^ use of th% resource, making ultimate dissem- 
ination on a broad scal^ less probable. It should be pointed out that 

extensive documentation of the lefssons themselves has occdrred, particularly 

' * V ' ^ * • 

in the case of elementary reading. Equally extensive orientation, training, 

and* support material are, however, ^lot yet in evidence. - ^ 

Thus, the ^majdl^ijjawback, from the evaluators' perspective, of JLmj^le- 

meriting close fo home is that it is difficult, \f not irresponsibly, to 

generait^ to less hot-house-like conditions, wheit the program must Svtand 

•■■ 

a^one without the facilitating presence of its own developers. For^ 
research and inquiry, when wider dissemination is not a Consideration « this 
can hfi a^viabrjle, even preferred choice* For a program slated for broad 
dissemination, it may still be appropriate to conduct field t^dls on 
familiar ground. Only vhen a field demonstration is intended to simuldte » 
the probable conditions of, future iioplementatioris would such a strategy 
be questionable^ . ^ * , , 

The choice of schools *wlthin a district has equally important effects. 
Schools may be chosen with a population of students and teachers that could 
unduly facilitate or hinder the acceptance of a program. The student* body 
may closely resemble or diverge widely from the target population'-iiC the 
program. The PLATO projects did not per se select schools as sites for 
introducing the program, but it must be acknowledged that the strategy of 
teacher selection did increase the likelihooH of certain schools, rather 
than others, becoming pilot sites. 

. 1 . .. 

• 0 - - 
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The selectjLon of .classrooms to house PLATO terminals was don^ ' 

indirectly, via the selection of teadhers. .A district-wide notice went \ . 

out ostensibly to all 'teachers (some principals were more thorough in 

transmitting information to tt^eir staff "than others) informing them of 

the program, and soliciting volunteers for. field trials. Relying on 

volunteers to make room in the class for programs in their early develop- 

mental phase is a common approach, reflecting assumptions, some of them' 

quite unexaminedl* about the teaching role, ''forces mptivating teachers,- 

the organization of schools, ahd more. ' ' 

♦ 

The basic assumption underlying the volunteer strategy is the 
importance of teacher commitment to the program. The developer wants 
the teacher to be on the program's side, investing it with positive - 
expectations or at least protecting it by suspended judgment/ The imple- 
mentor also wants the teacher to te willing to commit thie ♦effort th|^at 
*^ introducing the program requires, an effort which is often non trivial, 
irt'^olving understanding the develdpers* intentions, learning nev ins true- 
tional techniques, rethinking previously held constructs, and putting up 
with the frustrations that inevitably accompany the shakedown phases of any 
innovation. How to maximize the likelihood of these eonditions for imple- 
mentation? The inference s^ems reasonable that if a teacher ^comes forward, 
offering to gi?/e the program a home in her classroom, she is well motivated 
to explore i^s potential, and is likely to share the program's approach 
and goals.. 
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' On the face of it, an eminejAtly plausible, assumption. But in an 
extended intervi^, when probing teachers' reasons for volunteering to 
participate in the PLATO p^ject, we found a remarkably catholic set of 
motives, not all of which related to ^n interest in trying coiiq>uter-aid6d- 
instruction.iin the classroom. • A sizeable number of the teachers could be 
classified as "high innovators," i.e., teachers who have^a history of 
participating in a variety of new. projects . But there were a few 
teachers who had no affinity with the subject matter that vas to be computer- 
raided and hoped to be relieved by the computer of responsibility ior 
teaching it. Others hoped, that the compute^^ would prove helpfUl to 
particular children with whom the teacher felt^ ineffective. The teachers 
with the more intrinsic reasons for participating divided among those who 
felt in need of -help with their teaching of readj.ng or math, and those who 
were interested in learning about new approaches to instructix)n', seeing the 
program as an opportunity for professional growth, ^ere were teachers who 
volunteered for idiosyncratic reasona; one joined the program because it 
ensured her stay in the sa^ne school till retirement, others, wanted to 

' ' : } 

enlarge their community of interest in computers with a friend or spouse; 

House (1974) has suggested th^t the prospect of career advancement is a majoi 

• * *^ . 

* motivating force of entrepreneurs and early users of innovations. Among ^ 

the PLATO^eleroentary teachers, given the flat; c'^reer ladder' of elementary 

schools, this was a negligible factor, applicable mainly to the rare male 

teacher - incidentally putting into relief the f act ^that perceived dlff- 

erential opportunities for the sexes persist in these institutions. 



House, Ernest R. , Politlca of Education Innovation^ California: 
McCutchan, 1924. ^ 
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The strategy of selection by volunteerism, then, did not entirely 
serve its intended purpose — it did not assure the most receptive 
conditions for the program in its first time out of the laboratory. An 
alternative strategy, that o£ scanning the district for teachers with 
a special interest in the new instructional aid might have led to recruiting 

iL 

teachers with more relevant motivation. Thus, the appropriateness of 
a strategy is context-dependent, with the implementor needing both the 
foresight and a relatively clear understanding of priorities to make a 
reasoned decision. 

Yet another consequence of relying on volunteer practitioner^" was 
the foreclosure of independent choice of the pilot school^. Most of the 
"high-innovator" teachers were from two schools that have traditionally 
been hosts for programs emanating from the University, Although both 
schools had a cross section of SES represented in the student body, the 
chi;.dren of University faculty, and indeed of PLATO staff, were a conspicuous 
presence. The schools were also receptive to innovation, tolerant of the 
attendant disruptions, flexible regarding instruct^.onal styles,, and not . 
focused on a single mode of assessment, viewing achievement test results as ^ 

but one, and not necessarily the mo&t important, indicator of children s 
f f 

progress. 

^ The program subsequently- fanned out to other schools in the district, 
but only now, in ^its' third year, is the math program significantly present, 
in two schools that are largely composed^ of low-SES children, even 
though potential benefits to this population were identified early as a 
project goal. ^ 



f ' 
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^ - Orientation of.. Participants^ ^ * • ^ ' 
Thfe recruitment strategy gains significance whfen the question of 

teacher pre|)aration is "con fronted/ The developers need to define the 

• ' ' ' ' ^ / ' ^ > " " . ■ ' ' ■ " 

role that they expect the teachfers to play, they must consider the 

qualities of^ the teachers a^lready recrui ted assess , their working 

, etJLyironmeht?, its supports and constraints, and talJSk^ aqcount' 6f the 

implCTientation's own resources to bring about a favorable 'confluence of 

g^vens and aims, despite the invariably limited means for teacher "prien- 

tatioji'?-^ ' \ ' , \ ■ ' " 

It is ' a, rare^-implementor who apprpach'fes this t^ask with the"*regquisite 

hiumi.lity and wisdom. It is^ a rare "implemen tor, too, who^ gives the 

task Its fi^lX due. ' Few responsible programs nowadays flSglect it ^ ^ 

efttlrely, remembering the history of the golden afge of curriculx^i^, 

* ^ * ' - * If 

development projects of the 50's, when materials and teacher's guides 

'were often regarded as sufficient for curriculum improvement, andf the 

teaci^r's sensitive role was grudginj^ly acknowledged only when the expected 

improvement failed to materialize. , # a 

Views of the teacher s proper contribution, to program implementation 

vary widely among developer^, and interact significantly with the pedagogic 

notions underlying the materials. Programs with. narrowly defijied and 

circumscribed Use tend to provide teachers with a "script," expecting 

' only a faithful rendering of the prescribed behaviors^ Preparation 

for implementing such programs falls under the rubric of "training" 

rather than "education." Programs with greater flexibility of use entail 



13 



ERIC 



^ome teach©^ familiarity with program purposes, whLch presumably overlap 

with the Treacher's own xUrricular concemis. Thus • th^ developer, in his 

interactions with teacjiers, li^eds to take some account of their percep- ^ 

tions and conceptions' arid, if necessary, shape^ these to benefit thej. - . 

implementation. This process may be a short move away from^ training 

or a long one,, depending on thei^ complexity of the requisite* under^tandir^s , 

the teacher's entry state, and the weight' given to the Teachers' role. 

In the case of, richly and broadly .conceived programs, where a high level 

of teacher input is posited, developers are wq^t to select teachers with 

the desired quarlities, rather than face the uncertain prospect of educating. 

The two elementary projects differed somewhat in their views of and 

plans for teacher preparation. The math group was st^ffedlat the start 

of the Contract by a small core of former teachers with exteasive "-experience 
♦ 

in developing and implementing ah innovative mode "of math instruction. 
They came to the PLATO project with tested convictions about the teaching 
of ma^^hematics , ready to explore thd computer as a vehicle for the expression 
'of their precepts. 

The math staff had originally anticipated extensive sumnler wofrkshops 
for teachers, which would deal not only with the logistics and pragmatics 
of incorporating computer terminals into a classroom, but substantive 
questions in mathematics as well. These elaborate plans were not realized. 
The request for additional funding for teacher training was only partially 
successful, and the available project funds were not reallocated to cover 
the training effort.. Also, - teachers were reluctant to give up suiraner weeks. 



especially without acceptable reimbursement. 



Training for the math^^oiip evolved Into a varied set , of Inter- 
sections between PLATO staff and teachers* A .two day meeting was held 
In the s-unnner, . along with varied vlevels of self-scheduled wotklng at 
the termlpal by individual teachers. Throughout the year,f after 'school 
meetings were held with all the volunteer teachers. In addition, st|j^f 
members were available to spend, time at the terminal j^lth individual 
teacjiers, who used this opportunity Itf diverse ways. All went thKQugh 
some of the available less oris, and' learned how to access the student data 
kept by the system. A few explored some of the system's capabilities. 
MatK staff also spent considerable time in classrooms, with children and 
"te^achers, helping smooth the ^ransition,* obse1:vin-g the interaction of 
programs and children, troubleshooting for the frequent hardware and soft 
ware problems during the first months. ^ . 

The heading ^^roup represented a more diverse se,taof ba'ckgroXmds 
and .Interests. They shared some teaching experience, but ilot with a 
focus on ginning readers. The group's approach to the reading process 
was analytical, leading to the identification of a set of skills that , 
were assumed to be prerequisite "components of the ability to'read. The ^ 
programs were aimed at the development of these skills. Although the 
reading group expressed intentions of • forging these relatively dj.scr.ete 
skills into an Integrated model of the reading process as a resttlt of 
working with^ children on PLATO, during the flrst'Wave of teacher orienta- 
tion there was no perceived need to imbue teachers with a^partlcular view 



15 . ^ 
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^ of the pi?oces8 of learning to ^ « ' 

The reading and math s^aff. also differed in their relation to 
the eo^p>pter. The same members of the re'ading group |)l:ogrammed lessons 
as well a6 the course-specific control system — in the math group the 
router programming and curriculum design^unctions tended to be separated. 
The reading group on the whole had more interest in using the medium 
to its full potential, an(d conversely, in living within the limitations 
imposed by it. • *^ 

. The mode of teacher orienlxation was 'related to these predispo- 
sitions. A substantial amount of programming was invested by the 
reading staff in the preparation of on-line trafhing materials, which 
were intended to familiarize teachers with the relevant workings • and 
capacities of the system, as djta collector and s tore r^ diagnostic aide, 
as well as tutor. la addition, teachers were encouraged to go through 
the available lessons in the student mode. 

The programmed materials proved to be drastically underused. 
The reading group, much like the niatti group, found face- to-f ace inter- 
action with teachers and students the most effective and probably indi- 
spensable mode of orientation. As the majority of the core reading staff* 
worked almost exclusively on programming, ,two experienced persons Were* 
engaged for the* important task of classroom liaison. With the exception 
of one member of the reading team, a significant part of teacher and 
child orientation was carried out by new staff, who themselves had to be 
oriented to the program's rationale and intent. 
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Th^ overall picture of teacher orientation for both projects' 

If ■ " I 

summarizes to an uneven effort. Training traried froijv t^ache|r to teacher, 
as a result of the staff's desire to res'pond to individual nAeds and in- 
terests, and the consequences of the hazards of "early chaos. r 

The .quality and direction of the effort was consistent with thj^ 

developer's priorities, which centered- oa. curriculum design. Neither 

' ' ' " < 1 

1 

group under tbok broadly conceived teacher development, eveiji though each 
asserted the interrelationship between the use of • programmed materials 
and teaching modes. The resolution attempted by the math program was 
aimed at . attracting exemplary teachers, tl(e reading projsram, ' also 
working with volunteers, sought to* create programs that did not require 
extensive teacher involvement; 

Judgements on the effectiveness of Leacher orientation and the 
relationship between that process and the eventua\ deployment of the inno- 
vation are forbiddingly risky to make. A possible way to assess training 
procedures is to scan the intersecting areas betwee^ training foci and the 
intended audiences' concerns. Taking a first step in that direction, we 
distilled from the interviews the teachers' expectations regarding mode 
of PLATO use, and the anticipated benefits and apprehensions associated 
with the prospect of utilizing such a resource. We ordered the range 
of expectations into a tentative framework that may be used to place an 
indi;vidual teacher's expectations, or actual modes of .use,,^ as well as to 
summarize groups of teachers along the same dimensions. This framework is 
presented in Table I. 
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TABLE I. ^ 

Modes of PLATO Use, Anticipated/Actual 



A- POSITIVE E!kPECTATIONS/USES 
Instructlonal/sublect-^tter focus 



Intent in Use 


^ 1 

^ Tutorial/Expository 


Drill and Practice/Review 


Supplantive 


Supplemental 


Supplantive 


Supplemental 


Instructional 
(CAI) 


% 








Record Keeping/ 
Retrieval (CMI) 




• ■ \ 


N 


r 

— 


Enrichment ^ 










New Teaching 
Hod^ . ^ 


\ 






V — ' 



2.. Diagnostic - additional context for learning about students: 

1. for its own sake 

2. to lead to more effective' PLATO utilization 

3. to lead to better use of other resources 

4. more effective communication with parents 
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Instructional - additional focus 

1. motivation ^ 

2. acquis;ltion 9f good. work habits, increased attentiveness 

3. computer literacy 

4. medium for encouraging cooperation/helping among children 

5. sense of accomplishment resulting from controlling complex 

system 

Classroom management 

1 . reward • 

2. control 

3, isolation 

4, ° behavior shaping ^ » 
B, NEGATIVE EXPECtATIONS/USES 

.1. Distortion of child's conception of the nature of math 
^ or reading 

2. Fear of PLATO takeover ; 

3. Discipline^ problems 

4. Disruption of class routine 

5. Increased competitiveneOT 

6. Physical strain 

7. PLATO encoi^r^ging autistic trends 

. • 18 . 



The framework incorporates the whole range of expectations encountered 
among this group of teachers, most unfamiliar with computer-aided instruction, 
as well as impact anticipated by the developers ^an(d evaluators. Educationally 
trivial or unrealistic expectations were not screeched out of the list. 

The two-dimensional category related' to instructional use with "subject 
matter focus distinguishes between. supplantive and supplemental use. Neither 
programTaSv yet accepts supplantive responsibility, where instruction of a ^ 
segment of the curriculum is entrusted to the computer programs.' Supplemental 
use/ where' students interact with the programs in addition to their regular 
classroom instruction can subsume a wide range of expectations, and may be^ 
broken down further if the responses warrant .it. Both modes can be envisiohed 
as sexiving either tutorial, explicative use, where neV concepts or methods 
are introduced, or drill and practice, ailLmed at reinforcing concepts 
previously introduced to the student. 
^ ' Although the teachers differed in the degree to which they articula- 
ted suppositions, on the whole their ideas abt^t the nature of the resource^ 
they elected to try were vague and^ undifferentiated. A great diversity 
of expectations emerged among the teachers, which may be related to the early 
la<^ of firm information about the capabilities of the system and the'cha- 
ra^teristios of the prograife. The implications fo^tfaining are that if 
PLATO is the inkblot that it appeared to be to teacliers,' orientation will 
need to be broad indeed if it is to speak to all th*«concems of all those 
concerned. * ' 
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Disillusioned by the questionable ecological validity of laboratory 
research In learning, and by the lack of even .repllcablllty of tradltlplnal 
field educational research. In which tests and questlonnalj;^ separate the 
Investigator from the behavior under study, forming a wall of paper impermeable ^ 
to any but the strongest signal, a growing band of researchers is returning 
to the classroom to watch the behavior of interest Itself, It is fashionable 
to speak of the need for a phase of natural history before any further' theory^ 

"V, 

cons tiruct ion. - ^: ^ \ . - * 

^tvaluators are not Immune to this trend: Cronbach, Stake and others 
have emphasized a reportorial function for evaluation, and even those who 
demur at restricting evaluation to a descriptive role acknowledge -this Importance 
of process, as well as product (Cuba, 1975). 

While this recognition of the Importance of getting close to the 
phenomena is commendable, it is often accompanied by the hope that insight 
will automatically emerge frpm studying complete behavioral records. It won't. 
Simply counting unlntegrated behaviors without considering their contexts 
or arbitrarily iJnposing one's own context as a way of limiting the ran§e 
of behaviors considered Important ignores a central issue in the study of 
behavior: the problem of determining the functional unit^of behavior 
in contexts as experienced by the organism. We do not claim to here con- 
tribute to resolving that issue; we merely insist that it not be swept 
under the rug in discussions of observational techniques for evaluation. 

Sur^y the script and stage directions for a play constitute as complete 
a record of Surface behavior as any practicable observation* technique 



Cuba, E. G. Problems in Utilizing the Results of Evaluation. Journal 
of Research and Development in Education . 1975, £(3)^42^54. 
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could be expected to produce. Y^t actors spend far more tltoein trying to get 
Info their role, exploring the character's motivation and dynain:).^^! than in 
•Imply reproduc'ii£g^ lines and gestures. The n\imber of interpretations 
available of lago's- "behavior record" gives the lie to the new critics' 
hope that Internal evidence is sufficient to determine explanation- In spite ^ 
of this difficulty, to study of the staging of any project reguires attending 
the performances. ■ ^ C ' ) 

Having set the stage for the conf rxmtation of PLATO system, children 
and teachers, we take our seats and a^it curtain time, somewhat apprehensive 
at the prospett of following the action of a play in the naturalistic genre 
In whifch characters seldom explain their actions. 

Before this metaphor collapses under its own weight, let me point out 
that it represents an attempt to evoke some of the llm^atlons and f^strations 
Inherent in naturalistic 'observation as a technique for gathering evajLuation 
data. Observational methods can chronicle and rate pervasive^ patterns of 
overt behavior, but cannot get bc^yond the beharvior and into heads, where 
presumably the activity of major interest is taking place. For this reason, 
ve will attempt in later reports to* coordinate information gained from observa- 
tion with that provided by other data sources. ^ 

The results are not yet in, let alone coordinated, and despite the 
increasing pressures on evaluators to emulate^ the reigning drama critics 
by filing a judgment on opening night, we plan to continue to examine the 
dimensions on which judgments ought to be made and to pursue new sources of 
relevant information as the play continues to run. Rather, we'll try to 



^Aa Medley pointed out, to the extent that rare, "peak" experiences are 
determinants of outcome, they are unlikely to be captured by Intermittent 
classroom observations. 
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sketch the evolution of one of the ohservatlon Instniments helng employed 
to ohServe the effects of PLATO on the life of some twenty elementary 
classrooms, the conditions and hypotheses imder which it has been developed 
and used, and one Illustrative .result concerifing implementation. 

We assume we ai^^ preaching to the concerted on the issue of the importance 
of asseiasing degree and mod\ of implementation. A reasonable ODnsensus exists 
on the i|q)ossibility of Interpreting outcomes In ignorance of how or even ► 
whether "j:reatments" were, in fact, applied, A case can be made that the 
study of varieties and Impacts of appi^ches to confronting and Integrating, 
a potentially valuable but possibly demanding new reisource is most germane to 

policy, • ^ 

We see it as our responsibility to go beyond the function of £t CAI / 
consumers' report, provid^lng Information relevant to an administrator facing 
a decision to purchase or reject the system in its pi^sent fori. We must 
attempt also to identify the' issues and important determiners ol^ mode of 
use in the hope that when somfctday the developers of '^WITTGENSTEIN I" appear 
with ^n even m>re sophisticated technology, they will not be faced with 
reinventing the wheel In t^eir attempts to Implement their system to test 
its effectiveness in school situations. 

While it would be consistent with some traditions Ip ibvaluatioii to , 
lay out our a priori analysis of all questions important jfor the evaluation, 
a theory relating mode of Implementation to outcome, and to proceed to show\ 
how anfevaluation instrument was logically derived from these considerations, 
the aotual process^^s been considerably less antiseptic.' Bumping up agaiast 
the phenomena being evaluated continues to modify our dssessment of what should 
be looked at and how, 
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The observational approach employed in this evaluation was davelopad, 



during and in response to the less-than--optimal early in5)lein«itation, with 
Initial observations taking place during the first pilot tjssting of the 
systems in three ,classrooms one year ago/ fjie first o]>servatipns wefe the 
third author's attempts to characterize the teacfiers' classroom organizations 
and styles, which were later coupled with detailed narratives of individual 
childi;^'s behavior at the terminal. As these preliminary field reports 
arrived at ETS» they were studied and discussed in an attempt to identify 
'dimensions of variation and possibly significant indicators' of mode /iand 

\ 

effectiveness of use. Additional questions and areas of focus. were bug- 
gested by the observers and the Princeton staff and an iterative proc^^ss of 
instrument development <5egan. While' it was clear that design of a specific 
Instriunent, tailored to the features of the PLATO system was essential 
for the racording of individual cl^Lld interaction with the system, it had 

been hoped that an existing instrument fqr observation of the classroom as 
a whole would prove appropriate to the task of characterizing those 
variations in teacher approach that might affect mode of iioplementation, 
outcome^ and acceptance. None of the instruments catalog^ued in Mirrors 
for Bella vlor seemed to capture enough of the richness evident in the 
narrative reports that were being^ollected , but Ithe problem of reducing 
such material to manageable summaries required that a coding scheme be 
applied, either to the narratives themselves, at one remove from the 
phenomena under observation, or by the observers. Thus was born yet another 
observation checklist, deiaigned to supplement, but not to replace observers' 
running accounts. The early hope that a real->time checklist could be d^8ign«d 
which would obviate the need for most of the narrative « leave room for 

t 
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recording unantijiipafced events, and yet retain tl^ contextual Information 
necessary to assess the significance of a given behavior,^ proved optimistic. 

• The current truce» Wbrked out with the lively input of l;he two fields 
observers (the third andL fourth authors.) , Involves a postrobsprvation 
checklist designed to save some writing in the narrative, to provide 
opportunity for judgments of frequency of coinonly-seen behaviors, and for 
more global judgments of pervasive classroom style, but is not based on the 
assumption that the narrative can or should wither away. 

The development of the instrument for observing individual children 
(appendix A) at the terminal was a relatively straightforward process of 
creating a form to Record behaviors that were characteristically noted in 
narratives. This interaction is seen as a process to be observed for overt 
. Indicationa of the child's attitudes, comfort and ease of use, pace, Intarest 
task orientation, and understanding. To some extent, these represent 

evaluative dinensions in themselves, s[Q well as being causally linked to 
f 

aiBDunt of learning likely to be taking place. 

In the case of the observation of the clasdroom setting, the situation^ 
is less straightforward. Of the countless perspectives from which the 
transactions and activities of life in classrooms can be viewed, few con- 
sistent relationships among identifiable teaching styles or acts and pupil 
outcomes We emerged. Rosenshlne and Furst (1973) point out that although 

D 

earlier reviewers of classroom observation methodology and findings wrote 

"in the hope that observational instruments 
♦ would be used in correlational and. experimental^ 

studies where the criterion was student gain, 
and that such studies would Involve a cycle of 
probing and refinement which would Improve both 
instruction and sttidjent growth. Unfortunately 
this research has not been done to any great 
extent. Instead, as has been demonstrated, the" 



Rosenshlne, B., and Furst, If. The Use of Direct Observation to Study 
Teaching. In R.M.W, Travers (Ed.) Second Handbook of Research on Teaching. 
Chicago: Rand McN^lly, 1973. Pp./ 122-183. 



major use of observational Inbtmments has been^ 
_^ tcK describe , teaching and train teachers -In skills 
of \mdocumentated value. ^Such .activities are 
• n^c^sary^4»jt not sufficient'!, (p.. 162) 

We might question even the necessity of the latter of these two activities, 
Lut it ^seems to remain the case, as it was noted by Medley and Hitz^l a 
dozen years ago, that we don't yet kiiow how-to tell effective, teaching by 
looking at it. " — 

This assessment of the situation led us to an eclectic ,4i^pproach, with^ 
piloting of a number of existing instruments ^di^ to D. SolomoUf Soar and ^ 
Ragosta, Brown, and Trismen, Wilder, Nalln, Weinberg and Hardy. Previous 
experience in the analysis of a low-iixf erencg category instrument had led 
one member of the team to agree with Bosenshine and Furst that higher- 
inference sign, rating and global Judgment techniques of data recording offered 
the current best hope of adequate adaptation of the data recording process to 
the context of behavior. Our position is that at this stage of knowledge 
we shoiad build on the observational skij.l8 that IntQlligetlt human beings 
must possess to survive, (Heyns and Lippitt, 1954) rather than attempt to 
constrain them to simulate mechanisms with the limitations that go with the 
reliability of clockwork. ^ 

The process of trying, modifying or rejecting items from all of the 
above sources and even a few of our own, with much discussion and patient 
pointing out of gapa,\Jiard cases and Impossible distinctions by the on-site 
half of the team, led to a 155-item checklist reproduced in i^pendlx B. 

( ■ ■ ■ 

Heyns, R,. W., and liippit, R. Systematic Observational Techniques. In 
G. Llndzey (Ed.) Handbook of Social Psychology. Cambridge, Ma88> : Addison- 
Wesley, 1954^ Pp. 370-404. .' . 

26 



Built Into ifhe checklist, besides notations of certain physical ^ 
characteristics of the classroom, materials and curriculum approaches are 
several sets of Items designed to provide taeasures bf coastructs derived 
from previous research, eonstf&its related In specific hypotheses with which 
we approached the evaluation of these Implementation efforts^^ Among these 
are the constructs of Teacher, Behavioral Control, Teacher Cognitive Controll 
6r Input, Breadth vs. Narrowness of Focus, Degree of Pupil Coi^peratlon, and 
Teacher Involvement with PLATO. 



A major hypothesis relating tAese const^cts arose from the modes of 

implementation actually adopted. While a terminal room capable of handling 

Y . \ • - 

a whole class of children at a time was for a while contemplated at one 

high-innovating school with a large number of PLATO classes, the option, 

wfts rejected In favor of placing 1,2, or 3 terminals In each reading 

classroom, an'd 4 in most mathematics clasfllrooms. The consequence of this 

plke^ent strategy is that if allV children in a class^^re to have th€±T 

15 minute or half-hour turn on PLAT0**^ea5h day, some children must be using 

the terminals at almost all times. Thus we hypothesized that irrespective 

of the relationship of the curriculum to the teachers' goals, teachers high 

in attention to behavioral control^ particularly those who were accustomed 

to working with the whole class in a single group much of the d^, would find 

PLATO Intrusive and demanding without extensive in-service training in new 

modes of classroom organization. Since the traindf^g provided was focused on 

the mechanisms of system and lesson operation, with classroom organization 

aspects being left to the(^^enuity of Individual teachers, the hypothesis 

leads to the spedif ic predictions that teachers high on the control dimension 
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are likely to be low on PLATO acceptance while those more In tune with open, 
individualized, or even only "reading group" techniques would tend to be 
able to integrate ELATO terminals more easily Ijcito the congeries of resources 
they weri already managing. * ) 

A second hypothesis dealt with teacher's level of cognitive input, 
as distin^shed from befiavioral control. It was felt that nearly orthogonal' 
to the dimension of strict or loose classroom behavioral managen^nt shotuLd 
lie a^lmension of activism ot even instriisiveness with respect to the ^ 
children's learning. It was argued that the teacher high on this dimension 
wbuld be li^ly to evaluate PLATO lesscms carefully, and accept or reject 
the system in terms of its perceived educational value for individual childret|« 
*The delays that led the sequenc;^: graphing, whole numbers and then fractions 
in mathematics, an^ the fact that many first graders among the PLATO demon- 
stration classes enter knowing; how to read, was expected to lead such 
teachers to be critical of PLATO, if not to reject it for certain children. 

Finally because of the clear link of lesson materials to behavioral 
objectives corresponding to an analytic view of reading, and the much more 
free-flodting goals of many mathematics lessons, it was hypothesized that^ 
reading teachers having a "narrow focus" in Instiructlon would react more 
positively to the reading curriculirai, while those mathematics^"<eachers with* 
a narrow fcfcus (differently defined) in their instruction would react less, 
positively than would their counterparts with a broader conce^lon of ^children 
and curriculum. 

Preliminary examination of the results of the first two rounds of 
classroom observations (n » 38) completed in the fall of 1974, suggests that 
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some constructs fare better than others , and that While one hypothesl^ 
cannot be rejected, the others cannot yet be tested. ^^Idatl^ of these 
scales was carried out on the 57 observations of rounds 23, and- 4, and 
alphas and interscale correlations were found to remain quite stable. 

Tables 1-5 give the Items making up the control. Input, narrow foQus, 
and PLATO scales. Item-scale correlations, and coefficient alpha measxtres of 
conslstedcy of the scales formed of the sums of the standardized Items of 
the validation set. 

Insert Tables 1-5 about here v ^ 

As Is apparent from their reliabilities the scales varied in the degree 
to which they hang together, with PLATO involvement being the least well- 
measured construct. Nevertheless^, the reliabilities are acceptable if the 
scales are indeed measuring different things. 

Table 6 indicates that most of them are not. 

Insert Table 6 about here 

In^ particular. Scale 1, Teacher control, and Scale 2, teacher Input, 
Intercorrelate as high as their reliabilities permit, indicating^ that we\ 
have not succeeded in retrieving Independent measures of the two behavioral 
patterns, or possibly that in this sample at least, they are not In fact 
independent. The "narrow focus" scales relate to teacher input and consequently Intei 
relate more strongly than one -would prefer, although less strongly to behavioral ^ 
control. - « 

The correlation between the narrow focus scales in math and reading 
la of course spuriously Inflated by the fact thait thejy share items. Thus 
until the scales have been refined into more nearly individual entities, or, 
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as la more probable, the separate construct of teacKer "input" is abandoned 

and a>mblned with teacher control, we have no evidence that we are dealing 

with ©ore than one dimension. While the fact that teacher control relate slightly 

negatively (p<.15) to the.PLA^O involvement scale tends to support one of 

our original hypotheses* the possibility that ttils pattern of judgments 

reflects a bipolar, evaluative dimension contrasting high PLATO valors with 

"bad guys" cannot be discounted at this stage. To the extent that data 

from the teacher Interviews and system recotds of actual usage confirm 

this preliminary finding, and to the extent that the application of analyiilc - 

methods reveal a more differentiated* structure in these and later observations-, 

the hypothesis that we are chasing a halo effect will be Inflrmed. The 

narratives and observers* sunmarles suggest that observers do not attach any 

particular "halo" to PLATO use. If more thorough analyses support sub8tant|[ve 

findings in this area, their importance will be in the realm of refining 

the tautology that "traditional teachers" resist Innovation. Few of the 

volunteer teachers in this sample could be characterized as "tiradltlondl" 

in any simple sense of the wprd. The specific beliefs and behaviors that 

gp with a teachers' acceptance of the heavy demands inherent in maldng her 

classroom a proving ground for a new technology need to be understood if 

we are to hope to separate the potential of y^he play from the Idlosyncracles 

of the actors In any particular production. Classroom obseifvatlon, 

coordinated with interview and test data, shows promise In helping us in 

the task of clarifying and ultimately assessing this potential. 
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Individual Child at Terminal 
Observation Checklist 
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Teacher . 
Time q!>^ 



- 12 - 
School _^ 



J)ate 21- l5 



Mo/Day /Yr. 



Child's Name 



Observer 



' STUDENT INTERACTION WITH TERMINAL 
Except where Indicated otherwise, coding Is: 

1 • none or never 

2 - low level or seldom 

3 - medium level or sometimes 

4 - high level or often 

5 - extremely high level or constantly 
blank « no opportunity to observe 



Lesson Identiflc 


at ion ^oescrioe/ — 


^ TocVcLv\ \^ 





A. PROCEDURES AND CONTENT 



\ 



B 



1. * Child's understanding of directions 

2. * Child's understanding of content 

3: General Impression as to difficulty of lesson 
1 - too easy 3 - about right 5 - too hard 

4. Wliat do you tj>lnk was source of any difficulty 
child had with lesson? 











• 

— > 









lis:] CO CO [ZZ3 
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AFFECTIVE REACTION 

Non verbal expressions of attitude: 

5. Attention to PLATO terminal 

1 « no attention 5 - all attention 

6. 1 « bored 5 - highly involved 

7. 1 - tense^ 5 « relax^ 

8. 1 - discouraged 5 - confident 

.Verbal expressions of attitude while on PLATO: 
[1 - very negative 3 - neutral 5 « very positive] 

9. To self 

10. To other children 

11. To teachers 

12. To PLATO . ^ 















> 












5' 









^Coding elaboifated on attached sheets. 

Note;, A,B^C,D refer to separate lessons. If rating for item is same across 
lessons, rate only under lesson A and leave rest blank. 
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c/ RESPONSE STYLE • ' ^ 

[1 - child never acts this way 5 - child const^tly 

acts this way] 

13; Impulsive - makes response before looking or thinking 

14. Hes.itant 

15. Slow but confident 

16. Fast and accurate * . 

17. Self-motivated and purposeful -in approach 

18. Tries to "beat system" (touch;^ng happy face to 
go on without reading, giving wrong responses to. 
wait for machine to correct him, etc.) 

D. MECHANICAL J)IFFICULTIES . - 

19. * Child facility with typing 

20. * Child facility with managing audio device 

21. * Child facility with managing mictofiche 

22. System failure occurs 

l=never ^ 5=constantly 
Where in lesson?- 



23. Other hardware failures occur 

l«never 5=*c oris t ant ly 

What kind? . 



24. l-ength of wait for lesson clianges 

1 « none 3 - acceptable 5 - excessive 



E. REQUESTS FOR HELP ^ 

25. Proportion of time that C is assisted , 

1.= never 3 = several minutes 5 = constantly 

26. Request for help made to teacher 

' [l « never 3 = several times 5 =^ constantly] 

27. T responds by "doing for" 

28. T responds by guiding or" giving information 

29 . jj T acknowledges request but doesn't help 

30. Request made to other child(ren) 

31. C respond by "doing for" 

32. 0 C respond by guiding or giving information 

33. C acknowledge request but don't help 

3^. Request made to CERL staff member 

35. SAie responds by "doing for" 

36. , S/he responds by guiding or giving information 

37. S/he acknowledges request but doesn't help 

38. Request made to EXS observer 

• 33 
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Ijitoract'ion with child is initiated by: 

[l = never 3«several times 5*constantly J 

39. Teacher 

40. CERL staff 

41. Other adult 



A B C I> 



, 1 




V 


1 




t 


1 







p. OTHER CHILDREN AT JTEHMINAL 



/ 



42. Other children spend time around terminal 
Children comment on or talk to child at terminal 
about : 

43. PLATO procedures 

44. PLATO content ' 

45. unrelated matters 

46. Children Interact with child at terminal in 
diaruptive or interfering way 



II II I 



) 
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ELABORATION OF CODING 

'1. 1 - Child ha'a so much trouble understanding dlrectloira thmt 
s/he gives* up . « ^ 

2 - .Child has great difficulties In understanding directions 
« and asks for help. 

3 - Child. has some difficulty In imd^rstandlng directions, but 

gets- by. . ^ \ ' 

4 - Child has only a little difficulty understanding dlrsctlons. 

5 - Child follows directions quickly and confldsntly^ or even 

anticipates them. 

2. 1 - Child has so much trouble junders tending content that s/he* 
gives up. 

2 '-^hlld has great difficulty in understanding content and < 
asks for help . . * 

. » ^ ~ Child has some difficulty in understanding content, but gets 

Sv^ by without ' help. 

4 - Child has only a little difficulty understanding content. 

5 - Child grasps contipnt quickly and confidently. 

19. 1 - C has so much trotfble typing that s/he gives up. 

2 - C has so much trouble typing that asks for help. , 

3 - C -has a fair amount of difficulty typing but gets by. 
- C has only a little difficulty in typing. 

s. 

5 - C "^is xionfldent in typing, has no difficulty. ^ 

20. 1 - C has so much trouble operating audio (changing ^iscs , get ting 
discs, etc.) that s/he giveie up. / 

2 - C has so much trouble that s/he asks for help. 

3 - C has fmSfV amoun^t of trouble but gets by . 

4 - C has only a little difficulty. 

5 - C is confident in opera ting tfiudio , has no difficulty. 

21. 1 - d has'^so much trouble operating microfiche that s/he gives 
up • 

C has so much trouble that s/he asks for help. 

3 - C has fair amount of trouble but gets by 

4 - C hda only a little difficulty. 

5 - C is confident in operating microfiche, has no difficulty. 
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Appendix B ^ 
Classroom Observation Checklist 
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T«i«ch«r 



- 17 - 

School 



s/3/%r 



Mo/D«y/Yr 



ObaTv r . <.\ ] .^^l No. Chlldrsn 

CLASSROOM OiSERVAtlOM 

Cod«s (sxdspt vh«r« othsr coding Bpscifil^sd) 
l«nona or nmvmr 

2- ilow Isvttl or sttldoii ^ 

3- iBttdlUB 1«V«1 or SOBStlBSS 

4- ihlgh Ittval or oftsn s 

5- i€xtrttBttly high l«v«l or constantly 
blank«no opportunity to obaarva 

A. CLASSROOM SETTING 
1. Physical Sattlng (clrcla ona): 
a. Dasks - rows columns 
\^hj Dasks - Informal arrangaaant 



SI 000 I 















































J 











CdHSNMfll 





1 i 








—1 






V 


V 



















3. 



c. Balanca of desks (tablas) -f activity cantars 

d. Activity centers pradoalnata • 
Noise Level (clrcla one) i 

a. Quiet, chlldten working 

b. Quiet, but tense (teacher-enforced if'ather than spontanaoue) 

c. Eum of conversation 

d. Noisy, children working 

e/ Noisy, disruptive ^ <, 

f. Other (describe) ' 
Functional Us* of Spaca; chlldran vork at: 
Activity Can-tara * ■ 
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1 7. 






1 




8. 






y 




9. 




/ 




t 


10. 




J 




i 


11. 



4. Student desks 
3. Circle or table 
6. On the floor 

Movement of children^ 
Raise hand for peralsslon 
Go to teacher for hal|^ 
Move from student to student 
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U. MATERIAL KKSOURCES (chock Mny «pply) 
12. ' Tfxtbooks In use 

J • ■ 

JL3. Workbooks In use 

14. Other printed ■•tarlals In use (spac^fy) t 

15. Vi.u.l aids in use (ap^cif y) : J.^y • i vvV^oNiv c^^^ 

16. Concrete mstcrisls in use (specify): 

(e.g. cuisttnairc rods, balance acalaa, structured educational gaaaa) 
C. READING (check aa nany aa apply) ' 
Type of activity: 

17. Experience atoriea (children dictating) 

18. Word attack .akilla 



I li^ . Wo^d, meaning 

20. Oral reading 

21. Silent reading 

22. Comprehenaion exercizea 

23. Spelling, punctuation 

24. Writing 

25. Handwriting, copying 
Source of texta uaedt 

26 . Textbook/ workbook 

27. Chil^ aelected atoriea 
' 26. Child generated atoriea 

29 . Teacher generated atoriea 

30. Gawea, apecify: 

31. Referencea to PLATO or activitiea baaed on ^LA^TO sateriala 
D.t. MATH (check aa many aa apply) , 

Type of Activity { 



0 



3i.- Introduction of rulea by diacovery or inductive itpproach 

33. Introduction of rulep followed by exaaplea—deductive 

34. Introduction of concepta, principlea ^ 

35. Practicing operatlona, rulea**drill , uae of material 

36* Children aaked for illuatration\i of concepta (e.g., ahow addition 
on nuiiber, draw a picture of 1/2 and 1/4, etc.) 



38 



\ 
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Topic : 

37. W 

38. Fractions 

39. Daciaala ^ 
40« Graphins 

41. G«o«atry . 

42 . Writing opan aantancaa » aquationa 
43.. "Word problaaa*' 

44. ^ ^aauraaant 

45. Eatimation ^ 

46. MathenAtica Vocabulary (**aata," aub trahand » *' ate.) 







Congi 




J 











t/ 47; Othar! 1^ ' 

Sourca of problaaa: 

48. Taxtbook/workbook 

49. Child ganaratad pro^blaaa 
^ 50. Teacher ganaratad problaaa 

51. "^al-lifa" baaed on achool or. home environaant 

^ 52. Referencea to PLATO or activitiea baaed on PLATO Bateriala 

CLASSROOM ORGAMIZATIOM 

Teacher vorka with whole claaarooa 

In working with the whole claaa, who doea aOat of talking? (6ircle oni 

a. Teacher ^ 

b. Children 

c. Tether apenda about aa such tiae liataning Aa talking 
Teacher worka with aubgroupa 



53. 
54. 



55. 
56. 



In working with aubgroupa who doea aoat of talking? (cirAe on.) 

a. Teacher 

b. Children 

c. Teacher ap*enda about aa such tiae liataning aa talking 
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57. Toic^mr works with ilngle pupils 

58. In working with Individual pupils » who does noit of talking? (clrcla ona) 

a. Taachar * 

t>. Children ^ \ * 

Teacher spends about as auch time llst^enlng as telklng 

59. Seae tesk Is givsn for whole gcoup — children do not Interact "with ^ach oth 

60. Sane task Is given for whole group — with discussion* Interaction 

61. Verlety of ectlvltlss going on In subgroups 

62. Children cngegsd In Individual activities, not grouped 

63. Teacher dl-i^ects children to activities 

64. Children direct thaaselvss. but according. to schedule dlcteted by taachar 

65. Children direct theaselvss according to thalr own Interests In school work 

66. Children direct thansalvas according to social Motivation 

67. Shifts In activities or classrooa organl«atlon are accoapllshed (circle on 

(a^ reasonably snoothly 
b) In a d'lsruptlvs way 
F. MOTIVATION , COMTEOL 

Teacher nalntalns motivation + control byi 

68. Giving of privileges, prizes, gredes 

69. Loss of privileges 

70. Direct prelse 

71. Emphasizing Intrinsic value of lde#s or ectlvlty 

72. Reminding chlldrsn of rules \ 

73. Nsgativs statements or wernlnge 

74. Pointing out etudent(e) aa poeltlve model 

75. Pointing 6ut studsnt(s) as negative model 

76. Competition 
77 • Cooperet Ion 

78. Commands without rsasons for behevlor given 

79. Empheelslng reeeone^or behavior ^ 
so! Physlcel *contect poeltlve 
81. Physlcel contact negetlve 
BlK Isolating pupll(s) * 
83. Having pupil sit by teacher 
§4/ Having pupil stay after school 



■ ] 
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CLASSROOM ATMOSPHERE * ' / / 

85«»^ . Hardworking at«btph«r« 
86. Playful^ joking at«otph«r« 

87 • Casual atmosphara ^ ' 

88. Tansa atmosphfra 

89. Children ara discouragad or pravi(nted from expraaaing own experianca <f 

judgments 

90. Children express own erxperiancas and judgments 

H. DEVELOPMENT OF IDEAS ' ' 

91. Discussion, relatiocTs among ideas, inquiry as instructional technique, 

are emphasized 

92^* Memorilation, rote learning, as instructionel technique, ara amphaaisad 
93. Attention is concentrated on particular group of students (especially 
bright, aspecielly slow, noisy, boys, girla> 
^nly one answer is accepted as being correct 



9A. 
^5. 

96. 

97. 

98. 

99. 
100. 



Pupil is permitted to suggest Additional or alternative anawars 
Focus is on generalizations and understendings ,of structares or pattarn 
Focus is, on facts and rules 

Pupil is encouraged to experiment or try own idaea 
Topics or praa^et planft are narrowly adhered to 
Instruction is adjusted to student concerns and interests ^ 
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.S}01. Specific s^tep-by-step instructions ara given 
102. Guidelines ara given with some freedom of interpretation 
I . ACADEMIC EVALUATION . ; 

10^3. Teacher pasiaft judgment on p*s work (positive) ("Good") 
lOA. Teacher pasaas jiidgment on p*s work (negative) ("Bad") 
Teacher withholds judgment df p*a work 

106. Teactier imiMdiately reinforces p*s answer as "right" or "wrong" 

^ ^ ' 

107. Teacher has p decide when has been answered satisfactorily 

108. TcLacher; aaka another p^ to give answer if one p fails to answer quickly 

109. Teacher pri>^ides answer to p' who yieems confused or puzzled 

110. Taa^har givea p time to sit 9^ think, mull things over 

- . ^ 

■ ; 41 ' 
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Check: 





7 








ill. 




/ .. 








112. 


" « 










113. 
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114. 
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115. 












116. 
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V 
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117, 
118. 
119. 
120 * 
121. 
122. 
123. 
124. 



INTliKACTIONS WITH PLATO ■ 
TMch^r to«« to terminal to gat ln£pr««tion £ro« •y«te» on pupil ptttforwanct 
Teacher usar faadback from system to change p's assignment on PLATO 
Teacher uses feedback from system to group p's for special PLATO-related 

Instruction ox remediation 
P's arc expected to leave their actiVitlei unfinished when It's their turn c 
PLATO A 
's are expected to finish their activities before taking^.turn on PLATO 
Teacher disciplined p's at the terminal (tells them to be quiet, keeps p*s 

from interfering with other p*s) 
Teacher walks by terminals to observe p's work * 



Teacher helps p*s at the terminal 
Teacher uses child's turn at PLATO as reward or punishment 
Teacher restricts child's PLATO use for educational reasons 
Other children gather around p's at terminal 

Teacher posts schedule for PLATO use, + sxshedule is adhered to 
Teacher posts schedule for PLATO use, + schedule not" adhered to 
Teacher posts PLATO progress chart or other indication of how p'e* 
are doing on PLATO 



125. PLATO-related materials are present in the room, specify: 



Other children at terminals; mode of interaction: 

126. Involvixrr*work in cooperjTtive , interaction with p at terminal 

127. Helping-help p at terminkl^th problem 

) 128. Interfering-interact 16 n/gative way with p at terminal 

129. Controlling-take overV^Mltrol 

130. Socialixing-ititeract in social way 
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K. GLOBAL IMPRESSIONS (circle one number) ^ 

131.. PAcmc / . 

Relaxed 1 — 2 4 — 5- Ruehed . 

132. INVOLVBMENT * 

Abeorbed 4 — 3 — — 5 Bored 

^ / 

133. STUDENT RELATIONS ' 

e. Cooperative -1 — 2 ' 3,^ 4 — 5^ Competitive 

b. Supportive i — 2 — — ^ Critical 

134. REWARD STRATEGIES 

Approyal/Pcivilegee ^ — 3 {3j 4 — 5- Dieapproval/Pun/ehment 

135. TEACHER PSYCHOLOGICAL DISTANCE 

cloee i — 2 — ^^-4 — 5- Aloof 

136. STUDENT ABILITY TO FOLLOW INSTRUCTION 

Underetanding ^ (g) 2 4 5 Confueion 

137. CONTINUITY OF INSTRUCTION 

Sequenc^of Unrelated taelce i — 2 (2) 4 — ^ Thematic abeorption 
139. RULES- TEACHER 

Seldom Mentioned i — 2 3 U j 5 Frequently Cited 

139. RULES - STUDENT 

Many Apparent Rules * -1 — 2 — 3 V 4 ^ S Apparent Rulee 

140. CLASSROOM DECISlOtl MAKIIIG 

Centralized ^-^^ — ^ — ^ — ^ Decentralised 

141. TASK CHOICE 

S tuden^t Determined 1 — 2 — 9 — 4-^ ; Teacher Determ^ined 

142. STUDENT MOVEMENT 

Student DetermlMB* 4 — a^A3 — "Teacher Determined 

143. <^NDIVIDUAL ATTENTION ^ 

^ • High Kmphaeie 4 — 2 — 3— — 5^ Low Emphaeie 

144. PLATO INTEGilATION 

leolated Reeource 1 2 13)^ 4 — Integrated Reeource ^ 

145. Teacher retaine reeponeibility - 

for PLATO CONTROL 1 2 3 4 PLATO eeen ar reeponeible 

for fJLATO content 

146. Teacher coneieteney 



147. FLATO probleme dierupt 
other aietlvi^eei 



Stable ^-tVa — 3— »4 — St Erratic 

43 



. Never l - 2 3 ^ 4 — § Frequently 
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K. GLOttAL IMPRK8SI0NS CQiit'd (circl* ono number) , 
148* TEACHER ENCOURAGEMENT OF PLATO USE 

Low 4 2- — 3 ( i ^ — High 

149. TEACHER ENTHUSIASM . 

Flat -i 2 3 0 5- Gung-ho 

isO. CLARITY OF PRESENTATION 

if Low -i a- — 3 ^ frv High 

151. TASK ORIENTATION 

"Good Times" 



152. USE OF ORGANIZERS OR STRUCTURING 
COMMENTS (OVERVIEWS/SUMMARIES) 



■i 2 3 ( 4 J 5 "You're hett ~ to learn* 

Low 4 fl ' 3 ^ 5- 



153. INTELLECTUAL CHALLENGE 

Low -J^^=f 3 ( 4 / 5 High 

154, TEACHER SEEKS TO DISCOVER 

CHILD'S UNDERSTANDINGS 



^ il«ver -i 3 — ^ 5 Of 



7 

155. TEACHER TAKES INTO ACCOUNT 
CHILD'S UNDERSTANDINGS 



Mever -i 2 3 (^^f 3 Often 



WHEN AN ITEM IS MARKED IN THE MIDDLE OF THE SCALE BECAUSE OF A 
COMBINATION OF LOW & HIGH INSTANCES, FOLLOW SCALE BY AN ASTERISK 
AND ELABORATE IN NARRATIVE. 

SPECIAL CIRCUMSTANCE^ COMMENTS, & ELABORATIONS. 
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Table 1 

Observation Scale 1 Teacher Behavioral Controla 



Teacher and children observed In these actlvl- 
tles nore frequently than are other classes 



Item-scale 
Correlation 



53 
59 

63 
69 
72 
73 
78 
103 

131 
138 

142 



Teacher works with whole classroom 

Teacher gives same task to whole group 
no child-child Jait;eractlon 

Teacher directs children to activities 

Loss of privileges as a reward strategy 

Reminding children of rules 

Negative statements, or warnings 

Commands given without reasons for behavior 

Teacher passes judgment on pupils behavior 
or work 

Global assessment: pacing as more "rushed" 



Global assessment: 

cited" 

Global assessment: 
teacher determined 



rules as more "frequently 
student movement as more 



.56 

.70 
.45 
.35 
.48 

t 

.65 
.26 
.76 

.67 

.82 

.81 



Teacher and children observed In these actlvl- 
tlea less frequently than are other clasflea 



9* Children move from student to student 

10* Children move from activity to activity 

61* Variety of activities going on in subgroups 

62* Children engaged in Individual activities, 
not grouped 

65* Children direct themselves according to Interests 

105* Teacher withholds judgment of pupils' behavior 
or work |fe 



5-^ 



.80 
.75 
.'52 

.48 
.78 

.07 
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Table 1 continued 



Children gather around other pupils at 
terminals 

139* Global assessment: feir apparent rules for 
children 

140* Global assessment: classroom decision making 
as decentralized 



Alpha reliability of sum of standardized Items - .89 
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Table 2 

Observation Scale 2 Teacher Input 



Teacher and children relatively inore 
frequently observed In th^ae activltlea 

101 Specific step-by-^tep Instructions are given 

10^ Teacher Ismediately reinforces pupil's answer 

141 Global assessment of task choice as more 
teacher determined 



Item-scale 
Correlation 



.42 
.42 



.59 



Teacher ai^d children relatively leas 
frequently observed In these activities 



54* In working with whole class, children do 
most of talking 

56* In working with subgroups, children do most 
of talking 

57* Teaciher works with single pupils 

58* In working with Individual pupUa, child 
does most talking 

100* Instruction is adjusted to student concerns 
and Interests 

102* Guidelines are given with some freedom of 
Interpretation 

107* Teacher has pupil decide when question has 
been answered satisfactorily^ 

110* Teacher gives pupil time to sit and think, 
mull things over 



.34 

.38 
.45 

.24 

.60 

.61 

^63 

.62 



Alpha reliability of sum of standardized scores ■ .67 
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Table 3 

Scftle 3 Narrow Focua: Reading 



Teachet: and chlldreti relatively more 
frequently obaerved In theae actlvltlea 



Item-scale 
Correlation 



23 Spelling, punctuation 
26 Using textbook/vorkboolc 
92 



94 
97 
143 



Btphasls on memorization, rote learning, ar 
instruct ional technique 

Only one answer is accepted as being correct 

Focus is on facts and rulea 



Global judg;nent of lower 
individual attention 



iphasls on 



.05 
.42 

.72 
.55 
.70 

.42 



Teacher and children relatively less 
frequently obaerved in theae activities 



15* Visual aids in use 

16* Co^cret;e materials in use 

17* Experience stories (children dictating) 

29* Teacher-generated stories 

30* Games 

ft. 

90* Children express own experiences and 
judgments 

91* Bnphasls pn discussion, relationships among 
Ideas, inquiry as Instructional techniques 

95* Pupil is permitted to suggest additional or 
alternative answers 

96* Focus is on generalizations and understanding 
of structures or patterns 



.13 
.35 
.36 
.31 
.41 

.71 

.67 

^4 
.62 



Alpha - .75 
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Ta#Le 4 

Scale 4 Narrow Focua: Mathematlca 



Teacher and children relatively more 
freqnently observed In theae activities 

92 Emphasis on taeniorlzatlon , rote learning as 
Instructional technique 

94 Only one answer accepted as being correct 

97 Focus Is on facts and rules 

99 Topics or preset plans are narrowly adhered to 

143 Global Judgment of lower emphasis on Individual 
attention 

Teacher and children relatively less 
frequently obecrved in theae activities 

15* Visual aids In use 

32* Introduction of rules by discovery or Inductive 
approach 

36* Children asked for Illustrations of concepts 
44* Measurement as topic 
45* Estimation as topic 

49* Child-generated ptoblems | ^^^^^ textbook 

50* Teacher-generated problems > or workbook 

51* Real-life problems / 

90* Children express own experiences and Judgments 

91* Eaqihasls on discussion^ relationships among ideas, 
Inquiry as instructional techniques 

95* Pupil is permitted to suggest additional or 
alternative answers/^ 

96* Focus is on generalizations and tmder standing of 
. structures and patterns 

Alpha - .81 4 3 
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Table 5 

t 

Involvement with PLATO 



Teacher and children relatively iiore Item-scale 
frequently observed In theae activities Correlation' 

113 Teacher uses feedback from PLATO dystem to 
group pupils for special FLATO-related 

Instruction or remediation •SS 

116 Teacher disciplines pupils at PLATO terminals 

(tells them to be quiet, Iceepa children 

from Interfering with others' work) .47 

117 Teacher walks by terminals to observe pupils' \ 

work ( . .65 

118 Teacher helps pupils at the terminal .60 

144 Global assessment of PLATO as a more Integrated 

resource In the classroom •45 



Alpha " .41 
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TnVkl A A 
laDXe u , 






Halations among clasaroom observation 




(n ■ 57) 


TC , TI NPR 


OTM 


PLATO 


TC (.89) .68 .32 


.42 


-.18 


TI (.67) .59 


.54 


-.06 


NFR (.75) 


.78* 


-.05 


IBm 


(.81) 


-.13 


PLATO 




(.41) 



The main diagonal entries are coefficient alpha reliabilities. 
TC: Teacher control. TI: Teacher Input. NFR: Narrow focua 
In reading. NFM: Narrow focus In mathematics. *Spurlously 
high correlation becauae of overlapping Itema. 



